IMAGE SEARCH SYSTEM AND IMAGE SEARCH METHOD THEREOF 



BACKGROUND QF THE INVEN TI ON 
FIELD OF THE INVENTION 

5 The present invention relates to image search for 

searching for an image similar to a designated image 
based on the determination of similarities of an 
individual image and, more particularly, to an image 
search system for, even when feature of an image are 

10 represented by a different scheme, determining 

similarities of the image and searching for a similar 
image based on an image features amount according to 
each scheme, an image search method thereof and a 
storage medium which stores an image search program. 

15 DESCRIPT ION OF THE RELATED ART 

With feature of an image made into data and 
represented by an image features, image search has been 
conventionally conducted by determining similarities of 
each image using the image features and searching for a 

20 similar image (e.g. from among accumulated images), for 

which devices and computer programs have been developed 
These conventional image search techniques adopt 
a method of searching for a similar image mainly by 
using an feature which represents constituent of colors 

25 contained in an image (hereinafter referred to as color 

constituent features) to find similarities of the color 
constituent. 



Here, the color constituent features denotes an 
image features expressing color constituent of the image, 
by a representative color in the image and a constituent 
rate of the same, etc. and it can be also expressed by a 
5 frequency rate of each color contained in the image, or 

one or a plurality of representative colors and its or 
their color constituent rates, etc. 

The system using a color constituent features, 
however, has a problem that it is impossible to reflect 

10 structural feature of an image such as a shape and a 

positional relationship of objects in the image. 

Here, one example of conventional techniques for 
reflecting structural feature that an image has is 
disclosed in Japanese Patent Laying-Open (Kokai) No. 

15 Heisei 11-316819. The conventional technique recited in 

Japanese Patent Laying-Open (Kokai) No. Heisei 11-316819 
proposes a method of calculating a color constituent 
features contained in each block which is obtained by 
dividing an image into a plurality of blocks and using a 

20 set of values of the color constituent features of the 

respective blocks as a color distribution features for 
the determination and search of similarities. Since a 
color constituent features of each block is expressed in 
the color distribution features, structural feature of 

25 an image can be reflected in the determination of 

similarities . 

Disclosed in Japanese Patent Laying-Open (Kokai) 



No. 2000-259832 as another conventional method is the 
image search system using, as an image features, the 
frequency distribution features which represents a color 
arrangement of an image by a frequency region. 

Here, the frequency distribution features of an 
image denotes an image features which represents an 
energy of each band that is obtained by dividing a 
frequency distribution of a spectrum of a color (mean 
color) of an image into a plurality of bands and 
analyzing a frequency. 

Fig. 11 is a block diagram showing a structure of 
a main part of the image search system disclosed in 
Japanese Patent Laying-Open (Kokai) No. 2000-259832. The 
conventional image search system includes an image 
features generation unit 102 for generating a frequency 
distribution features 103 from applied image data 101, a 
frequency distribution features storage unit 104 for 
storing a frequency distribution features of an image to 
be searched in advance, and a similarity calculation 
unit 106 for calculating a similarity 107 using the 
frequency distribution features 103 of the applied image 
and an feature 105 stored in the frequency distribution 
features storage unit 104. Here, the image features 
generation unit 102 includes a reduced image generation 
unit 110 for generating a reduced image 111 from the 
image data 101, a frequency analysis unit 112 for 
conducting frequency analyses of the generated reduced 



image 111, and a DC component and partial AC component 
extraction unit 114 for extracting a DC component and 
partial AC component as the frequency distribution 
features 103 from among the frequency analysis unit 112 
5 and an orthogonal conversion coefficient 113. 

As introduced in " Text of WD 4.0 of MPEG-7 
Visual" (ISO/IEC JTC 1/SC29/WG11 N3521), one example for 
color distribution features could be a combination of 
the Grid Layout Descriptor and any of the Dominant Color, 
10 Color Structure and Scalable Color, and for frequency 

distribution features the Color Layout Descriptor is a 
typical example . 

As described in the foregoing, among examples of 
an image features are a color constituent features, a 
15 color distribution features and a frequency distribution 

features, use of which realizes determination of 
similarities of an image and image search. 

Other conventional technique is disclosed in 
Japanese Patent Laying-Open (Kokai) No. Heisei 09-237343 
20 proposes an image search system which in combination 

uses, a color histogram as a color constituent features 
among the above-described image features amounts, and an 
orthogonal conversion coefficient as a frequency 
distribution features. 
25 Fig. 12 is a block diagram showing a main part of 

the image search system disclosed in Japanese Patent 
Laying-Open (Kokai) No. Heisei 09-237343. The 



conventional image search system includes an input 
device 201 for receiving input of an image features, an 
image features storage unit 2 02 for storing an image 
features and a similarity calculation unit 203 for 
calculating a similarity between the image features 
applied through the input device 201 and the image 
features stored in the image features storage unit 202. 

The image features storage unit 202 includes an 
orthogonal conversion coefficient storage unit 204 for 
storing a frequency distribution features of an image 
and a color histogram storage unit 2 05 for storing color 
constituent information. 

The similarity calculation unit 203 includes an 
image features determination unit 206, a color histogram 
similarity calculation unit 207 and an orthogonal 
conversion coefficient similarity calculation unit 208. 
The image features determination unit 206 determines 
whether an image features 209 applied through the input 
device 201 represents color histogram information or an 
orthogonal conversion coefficient. When the image 
features determination unit 206 determines that the 
amount represents color histogram information, the color 
histogram similarity calculation unit 207 calculates a 
similarity 212 between an image features 210 composed of 
color histogram information which is output from the 
image features determination unit 206 and color 
histogram information 211 read from the color histogram 



storage unit 205. The orthogonal conversion coefficient 
similarity calculation unit 208 calculates a similarity 
215 between an image features 213 composed of orthogonal 
conversion coefficients which is output from the image 
features determination unit 206 and an orthogonal 
conversion coefficient 214 read from the orthogonal 
conversion coefficient storage unit 204 when the image 
features determination unit 206 determines that the 
amount represents an orthogonal conversion coefficient. 

Since for the system to conduct image search when 
a plurality of kinds of image features exist, it 
requires an image features amount data base and a 
similarity calculation means corresponding to an feature 
of an inquiry image, as many similarity calculation 
means and image features amount data bases as the number 
of kinds of image features should be provided, which 
makes the system complicated and manufacturing costs 
higher. 

In order to solve the problem, a conventional 
image search system which realizes image search by a 
simple device structure even when image search is 
conducted using both a color constituent features and a 
frequency distribution features is proposed. 

In the image search system, in a case where a 
device is internally provided with only a frequency 
distribution similarity calculation means, when an 
feature of an inquiry image or an image to be searched 



is a color constituent features, a mean color is 
calculated from the color constituent features and 
converted into a frequency distribution features to 
conduct similarity calculation based on the frequency 
5 distribution features. 

On the other hand, in a case where the device is 
internally provided with only a similarity calculation 
means for a color constituent features, when an feature 
of an inquiry image or an image to be searched is a 

10 frequency distribution features, the image is restored 

by inversely converting the frequency distribution 
features and then color constituent information of the 
restored image is extracted to conduct similarity 
calculation based on the obtained color distribution 

15 features. 

In a case where the system is provided with only 
a similarity calculation means for a frequency 
distribution features, when an feature of an inquiry 
image or an image to be searched is a color constituent 

20 features, a mean color is calculated from the color 

constituent features and the mean color is converted to 
a DC component of a frequency distribution features, 
thereby conducting similarity calculation based on the 
obtained frequency distribution features. 

25 As a result, determination and search of 

similarities of an image whose feature is represented as 
a color constituent features or a frequency distribution 



features can be executed only by the provision of a 
similarity calculation means for either one of the kinds. 

However, no method of alternately converting an 
image features has been realized between a color 
distribution features representing an image features 
which has an image structural feature and a frequency 
distribution features. 

As described above, the conventional devices have 
the following problems . 

First, since an image features indicative of 
feature of an image has various kinds, for the 
comparison and search of images, an image features of a 
kind common to both images to be compared (searched) 
should be prepared for the images. 

In addition, even when an image features amount 
of a kind common to both the images is provided, a 
function of conducting comparison and search based on 
the image features should be further provided in an 
image search system. For the image search system to 
realize comparison and search based on various kinds of 
image features amounts such as an amount of distribution 
feature and a frequency distribution features, it should 
be provided with a similarity calculation means based on 
each kind of image features amount, resulting in making 
the device complicated and increasing manufacturing 
costs . 

Secondly, conventional devices are incapable of 



comparing images whose features are represented by a 
color distribution features and a frequency distribution 
features, respectively. This is because alternate level 
conversion technique in practical use has not been 
realized between a color distribution features and a 
frequency distribution features. 

For example, in a case where an image features of 
an inquiry image is a frequency distribution features, 
it is impossible to search a data base which records a 
color distribution features of an image to be searched 
to find an image similar to the inquiry image. 

SUMMARY OF THE INVENTION 
An object of the present invention is to solve 
the above-described shortcomings of conventional art and 
to provide an image search system enables a simple 
device structure to realize comparison and search of 
each image whose feature is represented as a color 
distribution features or a frequency distribution 
features without deteriorating search performance, an 
image search method thereof and a storage medium which 
stores an image search program. 

Another object of the present invention is to 
solve the above-described shortcomings of the 
conventional art and to provide an image search system 
realizing comparison and search of each image regardless 
whether an image features of the image is a frequency 



distribution features of the image or a color 
distribution features of the same, an image search 
method thereof and a storage medium which stores an 
image search program. 
5 According to one aspect of the invention, an 

image search system for determining a similarity of an 
image whose feature are represented by either one of 
image features amounts, a color distribution features or 
a frequency distribution features, to search for a 
10 similar image, comprises 

means for converting, with respect to an image 
set to be a target whose kind of image features amount 
is to be changed among respective images to be searched 
and an inquiry image, a kind of image features amount of 
15 the image in question to make kinds of image features 

amounts of each the image to be searched and the inquiry 
image coincident with each other, and 

means for comparing the image features amount of 
the inquiry image with the image features amount of each 
20 the image to be searched based on the converted image 

features amount and determining a similarity of each 
image to search for a similar image. 

In the preferred construction, the image search 
system further comprises means for referring to data of 
25 the image features amount of each the image to be 

searched, and means for receiving input of data of the 
image features amount of the inquiry image. 



In another preferred construction, the image 
search system further comprises frequency distribution 
features conversion means for converting a frequency 
distribution features into a color distribution features 
indicative of feature similar to image features 
represented by the frequency distribution features in 
question, and color distribution similarity calculation 
means for comparing the color distribution features of 
the inquiry image with the color distribution features 
of each the image to be searched and determining a 
similarity of each image to search for a similar image, 
wherein the frequency distribution features conversion 
means renders all the kinds of image features amounts of 
each image to be searched and the inquiry image into the 
color distribution features. 

In another preferred construction, the image 
search system further comprises means for referring to 
data of the image features amount of each the image to 
be searched, means for receiving input of data of the 
image features amount of the inquiry image, frequency 
distribution features conversion means for converting a 
frequency distribution features into a color 
distribution features indicative of feature similar to 
image features represented by the frequency distribution 
features in question, and color distribution similarity 
calculation means for comparing the color distribution 
features of the inquiry image with the color 



distribution features of each the image to be searched 
and determining a similarity of each image to search for 
a similar image, wherein the frequency distribution 
features conversion means renders all the kinds of image 
features amounts of each image to be searched and the 
inquiry image into the color distribution features. 

In another preferred construction, the image 
search system further comprises frequency distribution 
features conversion means for converting a frequency 
distribution features into a color distribution features 
indicative of feature similar to image features 
represented by the frequency distribution features in 
question, and color distribution similarity calculation 
means for comparing the color distribution features of 
the inquiry image with the color distribution features 
of each the image to be searched and determining a 
similarity of each image to search for a similar image, 
wherein the frequency distribution features conversion 
means renders all the kinds of image features amounts of 
each image to be searched and the inquiry image into the 
color distribution features, and the frequency 
distribution features conversion means includes inverse- 
frequency transformation means for decoding an applied 
frequency distribution features to generate a decoded 
image, and color distribution features extraction means 
for extracting each pixel value of the decoded image as 
a color constituent features to extract a color 
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distribution features indicative of feature similar to 
image features represented by the applied frequency 
distribution features . 

In another preferred construction, the image 
search system further comprises means for referring to 
data of the image features amount of each the image to 
be searched, means for receiving input of data of the 
image features amount of the inquiry image, frequency 
distribution features conversion means for converting a 
frequency distribution features into a color 
distribution features indicative of feature similar to 
image features represented by the frequency distribution 
features in question, and color distribution similarity 
calculation means for comparing the color distribution 
features of the inquiry image with the color 
distribution features of each the image to be searched 
and determining a similarity of each image to search for 
a similar image, wherein the frequency distribution 
features conversion means renders all the kinds of image 
features amounts of each image to be searched and the 
inquiry image into the color distribution features, and 
the frequency distribution features conversion means 
includes inverse-frequency transformation means for 
decoding an applied frequency distribution features to 
generate a decoded image, and color distribution 
features extraction means for extracting each pixel 
value of the decoded image as a color constituent 
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features to extract a color distribution features 
indicative of feature similar to image features 
represented by the applied frequency distribution 
features . 

5 In another preferred construction, the image 

search system further comprises frequency distribution 
features conversion means for converting a frequency 
distribution features into a color distribution features 
indicative of feature similar to image features 

10 represented by the frequency distribution features in 

question, and color distribution similarity calculation 
means for comparing the color distribution features of 
the inquiry image with the color distribution features 
of each the image to be searched and determining a 

15 similarity of each image to search for a similar image, 

wherein the frequency distribution features conversion 
means renders all the kinds of image features amounts of 
each image to be searched and the inquiry image into the 
color distribution features, and 

20 the frequency distribution features conversion means 

includes inverse-frequency transformation means for 
decoding an applied frequency distribution features to 
generate a decoded image, image division means for 
dividing the decoded image into a predetermined 

25 plurality of blocks, and color distribution features 

extraction means for calculating each color constituent 
features of each the block to extract a color 
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distribution features indicative of feature similar to 
image features represented by the applied frequency 
distribution features. 

In another preferred construction, the image 
search system further comprises means for referring to 
data of the image features amount of each the image to 
be searched, means for receiving input of data of the 
image features amount of the inquiry image, frequency 
distribution features conversion means for converting a 
frequency distribution features into a color 
distribution features indicative of feature similar to 
image features represented by the frequency distribution 
features in question, and color distribution similarity 
calculation means for comparing the color distribution 
features of the inquiry image with the color 
distribution features of each the image to be searched 
and determining a similarity of each image to search for 
a similar image, wherein the frequency distribution 
features conversion means renders all the kinds of image 
features amounts of each image to be searched and the 
inquiry image into the color distribution features, and 
the frequency distribution features conversion means 
includes inverse-frequency transformation means for 
decoding an applied frequency distribution features to 
generate a decoded image, image division means for 
dividing the decoded image into a predetermined 
plurality of blocks, and color distribution features 



extraction means for calculating each color constituent 
features of each the block to extract a color 
distribution features indicative of feature similar to 
image features represented by the applied frequency 
5 distribution features. 

In another preferred construction, the image 
search system further comprises frequency distribution 
features conversion means for converting a frequency 
distribution features into a color distribution features 

10 indicative of feature similar to image features 

represented by the frequency distribution features in 
question, and color distribution similarity calculation 
means for comparing the color distribution features of 
the inquiry image with the color distribution features 

15 of each the image to be searched and determining a 

similarity of each image to search for a similar image, 
wherein the frequency distribution features conversion 
means renders all the kinds of image features amounts of 
each image to be searched and the inquiry image into the 

20 color distribution features, and the frequency 

distribution features conversion means includes inverse- 
frequency transformation means for decoding an applied 
frequency distribution features to generate a decoded 
image, image division means for dividing the decoded 

25 image into a predetermined plurality of blocks , and 

color distribution features extraction means for 
calculating each color constituent features of each the 
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block to extract a color distribution features 
indicative of feature similar to image features 
represented by the applied frequency distribution 
features, and the color distribution features extraction 
means determines a representative color of each the 
block obtained by the division by the image division 
means to extract a set of the representative colors as a 
color distribution features . 

In another preferred construction, the color 
distribution features extraction means calculates a 
color mean of a pixel in each the block obtained by the 
division by the image division means to determine a 
color of the calculated color mean as the representative 
color. 

In another preferred construction, the image 
search system further comprises color distribution 
features conversion means for converting a color 
distribution features into a frequency distribution 
features indicative of feature similar to image features 
represented by the color distribution features in 
question, and frequency distribution similarity 
calculation means for comparing the frequency 
distribution features of the inquiry image with the 
frequency distribution features of each the image to be 
searched and determining a similarity of each image to 
search for a similar image, wherein the color 
distribution features conversion means renders all the 
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kinds of image features amounts of each image to be 
searched and the inquiry image into the frequency 
distribution features. 

In another preferred construction, the image 
5 search system further comprises means for referring to 

data of the image features amount of each the image to 
be searched, means for receiving input of data of the 
image features amount of the inquiry image, color 
distribution features conversion means for converting a 
10 color distribution features into a frequency 

distribution features indicative of feature similar to 
image features represented by the color distribution 
features in question, and frequency distribution 
similarity calculation means for comparing the frequency 
15 distribution features of the inquiry image with the 

frequency distribution features of each the image to be 
searched and determining a similarity of each image to 
search for a similar image, wherein the color 
distribution features conversion means renders all the 
20 kinds of image features amounts of each image to be 

searched and the inquiry image into the frequency 
distribution features . 

In another preferred construction, the color 
distribution features conversion means comprises 
25 representative color determination means for determining 

a representative color of each bock in an applied color 
distribution features, image generation means for 
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generating an image which uses the representative color 
of each the block as a pixel, image size change means 
for changing the size of the image generated by the- 
image generation means to a predetermined size, and 
5 frequency distribution features extraction means for 

frequency-converting the image changed by the image size 
change means to extract a frequency distribution 
features indicative of feature similar to the image 
features represented by the applied color distribution 
10 features. 

In another preferred construction, each the image 
to be searched is set to be a target whose kind of the 
image features amount is to be converted, and the kind 
of image features amount of each the image to be 
15 searched is converted to be coincident with the kind of 

image features amount of the inquiry image. 

In another preferred construction, the inquiry 
image is set to be a target whose kind of the image 
features amount is to be converted, and the kind of 
20 image features amount of the inquiry image is converted 

to be coincident with the kind of image features amount 
of each the image to be searched. 

In another preferred construction, both the 
images, the inquiry image and the image to be searched, 
25 are set to be a target whose kind of the image features 

amount is to be converted, and the kinds of image 
features of the respective images, the inquiry image and 
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the each image to be searched, are converted. 

In another preferred construction, a circuit for 
converting a kind of image features amount of the 
inquiry image and a circuit for converting a kind of 
image features amount of each the image to be searched 
are provided independently. 

In another preferred construction, the color 
distribution features is set to be an image features 
amount obtained by dividing an image as an object 
representing feature into a predetermined plurality of 
blocks and determining a representative color of each 
the block to generate data of the representative color 
corresponding to each the block. 

In another preferred construction, the frequency 
distribution features is set to be an image features 
amount generated by converting an image as an object 
representing feature into a reduced image of a fixed 
size and subjecting the reduced image to frequency 
transformation . 

In another preferred construction, the frequency 
distribution features is set to be an image features 
amount generated by converting an image as an object 
representing feature into a reduced image of a fixed 
size and subjecting the reduced image to frequency 
transformation and quantization. 

According to another aspect of the invention, an 
image search method of determining a similarity of an 



image whose feature are represented by either one of 
image features amounts, a color distribution features or 
a frequency distribution features, to search for a 
similar image, comprising the steps of 

with respect to an image set to be a target whose 
kind of image features amount is to be converted among 
respective images to be searched and an inquiry image, 
converting the kind of image features amount of the 
image in question to make kinds of image features 
amounts of each the image to be searched and the inquiry 
image coincident with each other, and 

comparing the image features amount of the 
inquiry image with the image features amount of each the 
image to be searched based on the converted image 
features amount and determining a similarity of each 
image to search for a similar image. 

In the preferred construction, the image search 
method further comprising the steps of 

referring to data of the image features amount of 
each the image to be searched, and receiving input of 
data of the image features amount of the inquiry image. 

In another preferred construction, the image 
search method further comprises a frequency distribution 
features conversion step of converting a frequency 
distribution features into a color distribution features 
indicative of feature similar to image features 
represented by the frequency distribution features in 



question, and a color distribution similarity 
calculation step of comparing the color distribution 
features of the inquiry image with the color 
distribution features of each the image to be searched 
and determining a similarity of each image to search for 
a similar image, wherein the frequency distribution 
features conversion step renders all the kinds of image 
features amounts of each image to be searched and the 
inquiry image into the color distribution features. 

In another preferred construction, the image 
search method further comprises a step of referring to 
data of the image features amount of each the image to 
be searched, a step of receiving input of data of the 
image features amount of the inquiry image, a frequency 
distribution features conversion step of converting a 
frequency distribution features into a color 
distribution features indicative of feature similar to 
image features represented by the frequency distribution 
features in question, and a color distribution 
similarity calculation step of comparing the color 
distribution features of the inquiry image with the 
color distribution features of each the image to be 
searched and determining a similarity of each image to 
search for a similar image, wherein the frequency 
distribution features conversion step renders all the 
kinds of image features amounts of each image to be 
searched and the inquiry image into the color 



distribution features. 

In another preferred construction, the image 
search method further comprises a frequency distribution 
features conversion step of converting a frequency 
distribution features into a color distribution features 
indicative of feature similar to image features 
represented by the frequency distribution features in 
question, and a color distribution similarity 
calculation step of comparing the color distribution 
features of the inquiry image with the color 
distribution features of each the image to be searched 
and determining a similarity of each image to search for 
a similar image, wherein the frequency distribution 
features conversion step renders all the kinds of image 
features amounts of each image to be searched and the 
inquiry image into the color distribution features, and 
the frequency distribution features conversion step 
includes an inverse-frequency transformation step of 
decoding an applied frequency distribution features to 
generate a decoded image, and a color distribution 
features extraction step of extracting each pixel value 
of the decoded image as a color constituent features to 
extract a color distribution features indicative of 
feature similar to image features represented by the 
applied frequency distribution features . 

In another preferred construction, the image 
search method further comprises a step of referring to 



data of the image features amount of each the image to 
be searched, a step of receiving input of data of the 
image features amount of the inquiry image, a frequency 
distribution features conversion step of converting a 
5 frequency distribution features into a color 

distribution features indicative of feature similar to 
image features represented by the frequency distribution 
features in question, and a color distribution 
similarity calculation step of comparing the color 

10 distribution features of the inquiry image with the 

color distribution features of each the image to be 
searched and determining a similarity of each image to 
search for a similar image, wherein the frequency 
distribution features conversion step renders all the 

15 kinds of image features amounts of each image to be 

searched and the inquiry image into the color 
distribution features, and the frequency distribution 
features conversion step includes an inverse-frequency 
transformation step of decoding an applied frequency 

20 distribution features to generate a decoded image, and a 

color distribution features extraction step of 
extracting each pixel value of the decoded image as a 
color constituent features to extract a color 
distribution features indicative of feature similar to 

25 image features represented by the applied frequency 

distribution features . 

In another preferred construction, the image 
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search method further comprises a frequency distribution 
features conversion step of converting a frequency 
distribution features into a color distribution features 
indicative of feature similar to image features 
5 represented by the frequency distribution features in 

question, and a color distribution similarity 
calculation step of comparing the color distribution 
features of the inquiry image with the color 
distribution features of each the image to be searched 

10 and determining a similarity of each image to search for 

a similar image, wherein the frequency distribution 
features conversion step renders all the kinds of image 
features amounts of each image to be searched and the 
inquiry image into the color distribution features, and 

15 the frequency distribution features conversion step 

includes an inverse-frequency transformation step of 
decoding an applied frequency distribution features to 
generate a decoded image, an image division step of 
dividing the decoded image into a predetermined 

20 plurality of blocks, and a color distribution features 

extraction step of calculating each color constituent 
features of each the block to extract a color 
distribution features indicative of feature similar to 
image features represented by the applied frequency 

25 distribution features. 

In another preferred construction, the image 
search method further comprises a step of referring to 
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data of the image features amount of each the image to 
be searched, a step of receiving input of data of the 
image features amount of the inquiry image, a frequency 
distribution features conversion step of converting a 
5 frequency distribution features into a color 

distribution features indicative of feature similar to 
image features represented by the frequency distribution 
features in question, and a color distribution 
similarity calculation step of comparing the color 

10 distribution features of the inquiry image with the 

color distribution features of each the image to be 
searched and determining a similarity of each image to 
search for a similar image, wherein the frequency 
distribution features conversion step renders all the 

15 kinds of image features amounts of each image to be 

searched and the inquiry image into the color 
distribution features, and the frequency distribution 
features conversion step includes an inverse-frequency 
transformation step of decoding an applied frequency 

20 distribution features to generate a decoded image, an 

image division step of dividing the decoded image into a 
predetermined plurality of blocks, and a color 
distribution features extraction step of calculating 
each color constituent features of each the block to 

25 extract a color distribution features indicative of 

feature similar to image features represented by the 
applied frequency distribution features. 



In another preferred construction, the color 
distribution features extraction step determines a 
representative color of each the block obtained by the 
division by the image division step to extract a set of 
5 the representative colors as a color distribution 

features . 

In another preferred construction, the color 
distribution features extraction step calculates a color 
mean of a pixel in each the block obtained by the 

10 division by the image division step to determine a color 

of the calculated color mean as the representative color. 

In another preferred construction, the image 
search method further comprises a color distribution 
features conversion step of converting a color 

15 distribution features into a frequency distribution 

features indicative of feature similar to image features 
represented by the color distribution features in 
question, and a frequency distribution similarity 
calculation step of comparing the frequency distribution 

20 features of the inquiry image with the frequency 

distribution features of each the image to be searched 
and determining a similarity of each image to search for 
' a similar image, wherein the color distribution features 
conversion step renders all the kinds of image features 

25 amounts of each image to be searched and the inquiry 

image into the frequency distribution features. 

In another preferred construction, the image 



search method further comprises a step of referring to 
data of the image features amount of each the image to 
be searched, a step of receiving input of data of the 
image features amount of the inquiry image, a color 
distribution features conversion step of converting a 
color distribution features into a frequency 
distribution features indicative of feature similar to 
image features represented by the color distribution 
features in question, and a frequency distribution 
similarity calculation step of comparing the frequency 
distribution features of the inquiry image with the 
frequency distribution features of each the image to be 
searched and determining a similarity of each image to 
search for a similar image, wherein the color 
distribution features conversion step renders all the 
kinds of image features amounts of each image to be 
searched and the inquiry image into the frequency 
distribution features . 

In another preferred construction, the color 
distribution features conversion step comprises a 
representative color determination step of determining a 
representative color of each bock in an applied color 
distribution features, an image generation step of 
generating an image which uses the representative color 
of each the block as a pixel, an image size change step 
of changing the size of the image generated by the image 
generation step to a predetermined size, and a frequency 



distribution features extraction step of frequency- 
converting the image changed by the image size change 
step to extract a frequency distribution features 
indicative of feature similar to the image features 
5 represented by the applied color distribution features. 

According to a further aspect of the invention, 
an image search program for determining a similarity of 
an image whose feature are represented by either one of 
image features amounts, a color distribution features or 
10 a frequency distribution features, to search for a 

similar image by controlling a computer, comprising the 
functions of 

with respect to an image set to be a target whose 
kind of image features amount is to be converted among 

15 respective images to be searched and an inquiry image, 

converting the kind of image features amount of the 
image in question to make kinds of image features 
amounts of each the image to be searched and the inquiry 
image coincident with each other, and 

20 comparing the image features amount of the 

inquiry image with the image features amount of each the 
image to be searched based on the converted image 
features amount and determining a similarity of each 
image to search for a similar image. 

25 The image search system of the present invention 

decodes a frequency distribution features of an image to 
convert the amount into an image and extracts a 
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representative color from each block obtained by 
dividing the converted image into a plurality of blocks 
to generate a set of the colors as a color distribution 
features, thereby realizing conversion from the 
frequency distribution features to the color 
distribution features. In addition, the system 
calculates a mean color of each block from a 
representative color of each block contained in the 
color distribution features to generate an image having 
the mean color of each block as a pixel, magnifies or 
reduces the image and conducts frequency analyses of the 
image to generate a frequency distribution features, 
thereby realizing conversion from the color distribution 
features to the frequency distribution features. 

It is therefore possible, even when a color 
distribution features and a frequency distribution 
features exist as an image features, to determine 
similarities by alternately converting kinds of features 
to each other to unify a kind. 

As a result, search of a similar image can be 
realized irrespective of a kind of image features amount. 
In addition, a system whose similarity determination 
function is adapted only to either one of a color 
distribution features and a frequency distribution 
features is allowed to calculate and search for a 
similarity of an image of any kind of image features 
amount . 



Other objects, feature and advantages of the 
present invention will become clear from the detailed 
description given herebelow. 

TOIEF DESCRIPTION OF THE PRAWIN GS 

The present invention will be understood more 
fully from the detailed description given herebelow and 
from the accompanying drawings of the preferred 
embodiment of the invention, which, however, should not 
be taken to be limitative to the invention, but are for 
explanation and understanding only. 

In the drawings : 

Fig. 1 is a flow chart for use in explaining 
processing of an image search system according to one 
embodiment of the present invention; 

Fig. 2 is a block diagram showing a structure of 
an image search system according to a first embodiment 
of the present invention; 

Fig. 3 is a flow chart for use in explaining 
processing of a frequency distribution features 
conversion unit in one embodiment of the present 
invention; 

Fig. 4 is a block diagram showing a structure of 
an image search system according to a second embodiment 
of the present invention; 

Fig. 5 is a block diagram showing a structure of 
an image search system according to a third embodiment 
of the present invention; 
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Fig. 6 is a block diagram showing a structure of 
an image search system according to a fourth embodiment 
of the present invention; 

Fig. 7 is a flow chart for use in explaining 
5 processing of a color distribution features conversion 

unit in one embodiment of the present invention; 

Fig. 8 is a block diagram showing a structure of 
an image search system according to a fifth embodiment 
of the present invention; 
10 Fig. 9 is a block diagram showing a structure of 

an image search system according to a sixth embodiment 
of the present invention; 

Fig. 10 is a block diagram showing a structure of 
an image search system according to other embodiment of 
15 the present invention; 

Fig. 11 is a block diagram showing a main part of 
a structure of a conventional image search system; 

Fig. 12 is a block diagram showing a main part of 
a structure of a conventional image search system. 

20 

PESCRIPTIQtJ O F THE PREFERRED EMBQPIMENT 
The preferred embodiment of the present invention 
will be discussed hereinafter in detail with reference 
to the accompanying drawings. In the following 
25 description, numerous specific details are set forth in 

order to provide a thorough understanding of the present 
invention. It will be obvious, however, to those skilled 
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in the art that the present invention may be practiced 
without these specific details. In other instance, well- 
known structures are not shown in detail in order to 
unnecessary obscure the present invention. 
5 Fig. 1 is a flow chart for use in explaining 

processing of an image search system according to one 
embodiment of the present invention. 

In the example of image search processing shown 
in Fig. 1, first, the image search system receives input 

10 of an image features of an inquiry image which is an 

image whose search is inquired and refers to data of an 
image features of each image to be searched (Steps 101, 
102). Here, the kind of image features of each image is 
either a color distribution features or a frequency 

15 distribution features. 

Here, since when the inquiry image and each image 
to be searched differ in a kind of image features, 
determination or search of similarities is impossible, 
set kinds of both image features to be coincident with 

20 each other by converting the kinds of the image features 

(Step 103) . 

Then, using the converted image features, compare 
the image features of the inquiry image with an image 
features of each image to be searched to calculate a 
25 similarity (Step 104) and output a calculation result of 

the similarity (Step 105). 

Here, data of an image to be searched is obtained, 
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for example, by referring to data stored in advance in a 
storage unit such as a hard disc or a semiconductor 
memory or to data stored in an external server through a 
communication network. 
5 Although in the example of Fig. 1, reception of 

an inquiry image and reference to an image to be 
searched are conducted according to an image features 
imilarly executable is a manner of accepting (or 
referring to) image data as it is and extracting an 

10 image features from the image data to conduct Step 103 

and the following steps. 

The method of alternately converting image 
features between a color distribution features and a 
frequency distribution features at Step 103 will be 

15 described in detail in each of the following embodiments. 

In addition, although in the example shown in Fig. 
1, the processing ends up with output of a similarity 
calculation result, it may end up with output of an 
image whose similarity is calculated to be high as a 

20 research result as well. Moreover, other than 

calculating similarities of a plurality of images to be 
searched to an inquiry image, processing of outputting 
data of a similarity of one image to be searched to an 
inquiry image is executable as well. 

25 [FIRST EMBODIMENT] 

Fig. 2 is a block diagram showing a structure of 
an image search system according a first embodiment of 



the present invention. 

With reference to Fig. 2, the image search system 
of the present embodiment includes a to-be-searched 
image frequency distribution features storage unit 62, a 
5 frequency distribution features conversion unit 20 and a 

color distribution similarity calculation unit 41. The 
frequency distribution features conversion unit 20 
includes an inverse-frequency transformation unit 21, an 
image division unit 22 and a color distribution features 

10 extraction unit 23. 

The to-be-searched image frequency distribution 
features storage unit 62 stores a frequency distribution 
features of an image which is an image features of an 
image to be searched. Here, a frequency distribution 

15 features of an image represents an feature indicative of 

a frequency distribution of the image, one example of 
which is a value indicative of an energy of each band 
obtained by converting an image to have a fixed size, 
dividing a frequency distribution of a color spectrum of 

20 the converted image into a plurality bands and executing 

frequency analyses of the bands using an orthogonal 
matrix such as DCT (Discrete Cosine Transform). The 
frequency distribution features may be stored after 
being subjected to quantization. 

25 Fig. 3 is a flow chart for use in explaining 

processing of the frequency distribution features 
conversion unit 2 0 according to the present embodiment. 



In the frequency distribution features conversion 
unit 20, first, the inverse-frequency transformation 
unit 21 inverse-frequency-converts a frequency 
distribution features read from the frequency 
5 distribution features storage unit 62 to generate and 

output a decoded image (Step 301). As inverse-frequency 
transformation, inverse discrete cosine transform (IDCT) 
is conducted to restore an image in a case where 
discrete cosine transformation (DCT) is used as a 

10 frequency analysis method at the time of generating a 

frequency distribution features. 

Subsequently, the image division unit 22 divides 
the applied decoded image into a fixed plurality of 
blocks (Step 302 ) . 

15 Then, the color distribution features extraction 

unit 2 3 determines a color constituent features of each 
of the blocks obtained by the division by the image 
division unit 22 to extract a set of the amounts as a 
color distribution features (Step 303). Here, the color 

20 constituent features representing color constituent of 

each block is an feature which is composed of a 
representative color representing each block of the 
image and its constituent rate and which may include a 
frequency rate of every color contained in each block, 

25 one or a plurality of representative colors and its or 

their color constituent rates or the like. 

The color distribution similarity calculation 



unit 41 calculates a similarity between a color 
distribution features of an inquiry image applied as the 
image features for use in inquiry and the color 
distribution features generated by the frequency 
5 distribution features conversion unit 20. 

Next, processing of the image search system 
according to the present embodiment will be described 
with respect to a specific example. 
M: First, with respect to each image to be searched, 

5 10 after dividing the image into, for example, 8x8 = 64 

blocks, calculate a mean value (of color, pixel) of each 
fji individual block to generate a reduced image having 8 

% pixels by 8 pixels. Next, subject the reduced image to 

* DCT to regard an obtained coefficient as a frequency 

fy 15 distribution features of the original image. By this 

method, a frequency distribution features of each image 
C3 to be searched is extracted and stored in advance in the 

frequency distribution features storage unit 62. 

Here, a mean of a color denotes a mean value of 
20 numeric data in a case where color data is expressed by 

various kinds of methods such as RGB color space, YCbCr 
color space and HSV color space . When each of the 
various kinds of color spaces has a unique method of 
calculating a mean of a color, the mean value will be 
25 obtained by such a method. 

Also with respect to an inquiry image, after 
dividing the image into, for example, 4x4 = 16 blocks, 



extract a color constituent features composed of data of 
a representative color which represents each individual 
block and its constituent rate to assume a set of data 
of correspondence between each block and its color 
constituent features to be a color distribution features 
of the inquiry image. 

Here, for the search for an image whose feature 
is similar by using the color distribution features of 
the inquiry image, data of the frequency distribution 
features stored in the frequency distribution features 
storage unit 62 will be converted into a color 
distribution features and both images will be compared 
using their color distribution features to calculate 
similarities . 

For this purpose, the image search system first 
conducts inverse discrete cosine transform (IDCT) with 
respect to the frequency distribution features read from 
the to-be-searched image frequency distribution features 
storage unit 62 to generate an image of 8 pixels by 8 
pixels . 

Next, after dividing the obtained image of 8 
pixels by 8 pixels into 4 x 4 = 16 blocks, extract a 
color constituent features composed of a representative 
color representing an individual block and its 
constituent rate. 

As a result, since the inquiry image and an image 
to be searched can be compared with respect to the 
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feature of the same dimension, search is realized by 
comparing a color distribution features which is a set 
of extracted color constituent features and the color 
distribution features of the inquiry image and 
5 calculating their similarities. 

By thus sequentially reading frequency 
distribution features of a plurality of images to be 
searched which are stored in the to-be-searched image 
frequency distribution features storage unit 62 and 

10 calculating a similarity between a color distribution 

features obtained by converting the read frequency 
distribution features and the color distribution 
features of the inquiry image to select an image to be 
searched according to the degree of the calculated 

15 similarity, an image similar to the inquiry image can be 

specified from among the plurality of images to be 
searched whose frequency distribution features is stored 
in the to-be-searched image frequency distribution 
features storage unit 62 . 

20 As described in the foregoing, the image search 

system of the present embodiment enables calculation of 
a similarity between a color distribution features 
obtained by converting a frequency distribution features 
which is an image features and an applied color 

25 distribution features. Moreover, since both a color 

distribution features and a frequency distribution 
features can be handled by the same similarity 
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calculation unit, the structure of the search system can 
be scaled down. 

Furthermore, an image search system can be 
provided which is capable of conducting search 
5 regardless whether an image features is a color 

distribution features or a frequency distribution 
features - 

[SECOND EMBODIMENT] 

Fig. 4 is a block diagram showing a structure of 

10 an image search system according to a second embodiment 

of the present invention. With reference to Fig. 4, the 
image search system of the present embodiment includes 
the to-be-searched image frequency distribution features 
storage unit 62, frequency distribution features 

15 conversion units 20 and 20a and the color distribution 

similarity calculation unit 41. The frequency 
distribution features conversion units 20 and 20a 
include inverse-frequency transformation units 21 and 
21a, image division units 22 and 22a and color 

20 distribution features extraction units 23 and 2 3a, 

respectively. 

The to-be-searched image frequency distribution 
features storage unit 62 stores a frequency distribution 
features which is an image features of an image to be 

25 searched. 

In the frequency distribution features conversion 
unit 2 0 on the side of the frequency distribution 



features storage unit 62, the inverse-frequency 
transformation unit 21 inverse-frequency-converts a 
frequency distribution features read from the frequency 
distribution features storage unit 62 to output a 
5 decoded image. The image division unit 22 divides the 

applied decoded image into a plurality of blocks. The 
color distribution features extraction unit 23 generates 
a color constituent features from each divisional block 
obtained by the division by the image division unit 22 
10 to extract a set of the amounts as a color distribution 

features . 

In a case where the number of blocks (vertical 
and horizontal) to be divided by the image division unit 
22 coincides with the number of pixels (vertical and 

15 horizontal) of the decoded image itself, the image 

division unit 22 has no need of dividing the decoded 
image. In this case, the color distribution features 
extraction unit 23 only needs to extract each pixel 
value of the decoded image as a color constituent 

20 features. 

In the frequency distribution features conversion 
unit 2 0a on the side of receiving input of an inquiry 
image, the inverse-frequency transformation unit 21a 
inverse-frequency-converts a frequency distribution 

25 features of the inquiry image applied as the image 

features for use in inquiry to output a decoded image. 
The image division unit 22a divides the decoded image 
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into a plurality of blocks. The color distribution 
features extraction unit 23 generates a color 
constituent features from each block obtained by the 
division by the image division unit 22a to extract a set 
5 of the amounts as a color distribution features. 

The color distribution similarity calculation 
unit 41 calculates a similarity between the color 
distribution features generated by the frequency 
distribution features conversion unit 20 and the color 

10 distribution features generated by the frequency 

distribution features conversion unit 20a. 

By thus sequentially reading the frequency 
distribution features of the plurality of images to be 
searched which are stored in the to-be-searched image 

15 frequency distribution features storage unit 62 to 

calculate a similarity between a color distribution 
features obtained by converting the read frequency 
distribution features and a color distribution features 
obtained by converting the frequency distribution 

20 features of the inquiry image, thereby selecting an 

image to be searched according to the degree of the 
calculated similarity, an image similar to the inquiry 
image can be specified from among the plurality of 
images to be searched whose frequency distribution 

25 features is stored in the to-be-searched image frequency 

distribution features storage unit 62. 

As described in the foregoing, the image search 
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system of the present embodiment enables calculation of 
a similarity between a color distribution features 
converted from a frequency distribution features which 
is an image features and a color distribution features 
5 converted from an applied frequency distribution 

features. It is therefore possible, also in a case where 
only the color distribution similarity calculation unit 
41 is provided without the provision of a frequency 
distribution similarity calculation unit, to compare 

10 frequency distribution features . 

Moreover, an image search system can be provided 
which is capable of conducting search regardless whether 
a kind of image features is a frequency distribution 
features or a color distribution features. 

15 [THIRD EMBODIMENT] 

Fig. 5 is a block diagram showing a structure of 
an image search system according to a third embodiment 
of the present invention. With reference to Fig. 5, the 
image search system of the present embodiment includes a 

20 to-be-searched image color distribution features storage 

unit 61, the frequency distribution features conversion 
unit 20a and the color distribution similarity 
calculation unit 41. The frequency distribution features 
conversion unit 20a includes the inverse-frequency 

25 transformation unit 21a, the image division unit 22a and 

the color distribution features extraction unit 23a. 

The to-be-searched image color distribution 
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features storage unit 61 stores, as an image features of 
an image to be searched, a color distribution features 
which is a set of color constituent features of the 
respective blocks obtained by dividing the image to be 
5 searched into a plurality of blocks. 

In the frequency distribution features conversion 
unit 2 0a, the inverse-frequency transformation unit 21a 
inverse-frequency-converts a frequency distribution 
features of an inquiry image applied as an image 

10 features for use in inquiry to output a decoded image. 

The image division unit 22a divides the applied decoded 
image into a plurality of blocks. The color distribution 
features extraction unit 23a generates a color 
constituent features from each block obtained by the 

15 division by the image division unit 22a to extract a set 

of the amounts as a color distribution features. 

In a case where the number of blocks (vertical 
and horizontal) to be divided by the image division unit 
22 coincides with the number of pixels (vertical and 

20 horizontal) of the decoded image itself, the image 

division unit 22a has no need of dividing the decoded 
image. In this case, the color distribution features 
extraction unit 2 3a only needs to extract each pixel 
value of the decoded image as a color constituent 

25 features. 

The color distribution similarity calculation 
unit 41 calculates a similarity between the color 



distribution features generated by the frequency 
distribution features conversion unit 20a and a color 
distribution features read from the to-be-searched image 
color distribution features storage unit 61. 
5 By thus sequentially reading color distribution 

features of the plurality of images to be searched which 
are stored in the to-be-searched image color 
distribution features storage unit 61 to calculate a 
similarity between the color distribution features and 

10 the color distribution features obtained by converting 

the frequency distribution features of the inquiry image, 
thereby selecting an image to be searched according to 
the degree of the calculated similarity, an image 
similar to the inquiry image can be specified from among 

15 the plurality of images to be searched whose color 

distribution features is stored in the to-be-searched 
image color distribution features storage unit 61. 

As described in the foregoing, the image search 
system of the present embodiment enables calculation of 

20 a similarity between a color distribution features 

converted from a frequency distribution features of an 
applied image and a color distribution features which is 
an image features. This allows both of the color 
distribution features and the frequency distribution 

25 features to be handled by the same similarity 

calculation unit to realize reduction in scale of the 
search system structure. 
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Moreover, an image search system can be provided 
which is capable of conducting search regardless whether 
a kind of image features is a frequency distribution 
features or a color distribution features. 
5 [FOURTH EMBODIMENT] 

Fig. 6 is a block diagram showing a structure of 
an image search system according to a fourth embodiment 
of the present invention. With reference to Fig. 6, the 
image search system of the present embodiment includes 
10 the to-be-searched image color distribution features 

storage unit 61, a color distribution features 
conversion unit 30 and an frequency distribution 
similarity calculation unit 42. The color distribution 
features conversion unit 30 includes a representative 
15 color determination unit 31, an image generation unit 32, 

an image size change unit 33 and a frequency 
distribution features extraction unit 34. 

The to-be-searched image color distribution 
features storage unit 61 stores, as an image features of 
20 an image to be searched, a color distribution features 

which is a set of color constituent features of the 
respective blocks obtained by dividing the image to be 
searched into a plurality of blocks. 

Fig. 7 is a flow chart for use in explaining 
25 processing of the color distribution features conversion 

unit 30 of the present embodiment. 

In the color distribution features conversion 
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unit 30, the representative color determination unit 31 
determines a representative color of each block based on 
color constituent information of each block included in 
the color distribution features read from the to-be- 
searched image color distribution features storage unit 
61 (Step 701). Here, applicable as a method of 
determining a representative color are calculating a 
mean value of a color contained in the color constituent 
features by multiplying each color by a rate of the same 
and adding all the multiplication results, obtaining a 
mean value by converting each color into a different 
color space (e.g. RGB color space, YCbCr color space and 
HSV color space) and other various methods using a mode, 
a medium value etc. as a representative color. The image 
generation unit 32 generates an image composed of the 
representative colors of the respective blocks 
calculated by the representative color determination 
unit 31 (Step 702) . 

The image size change unit 33 changes the size of 
the image generated by the image generation unit 32 to 
have a predetermined size (Step 703). The image size 
obtained after the conversion can be made coincident 
with the size of the original image at the generation of 
the frequency distribution features. 

The frequency distribution features extraction 
unit 34 frequency-converts the image obtained by the 
image size change unit 33 to extract the frequency 
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distribution features (Step 704). 

The frequency distribution similarity calculation 
unit 42 calculates a similarity between a frequency 
distribution features of an inquiry image applied as the 
5 image features for use in inquiry and the frequency 

distribution features generated by the color 
distribution features conversion unit 30. 

By thus sequentially reading color distribution 
features of the plurality of images to be searched which 

10 are stored in the to-be-searched image color 

distribution features storage unit 61 to calculate a 
similarity between an orthogonal conversion coefficient 
obtained by converting the read color distribution 
features and an orthogonal conversion coefficient of the 

15 inquiry image, thereby selecting an image to be searched 

according to the degree of the calculated similarity, an 
image similar to the inquiry image can be specified from 
among the plurality of images to be searched whose color 
distribution features is stored in the to-be-searched 

20 image color distribution features storage unit 61. 

Next, processing of the image search system 
according to the present embodiment will be described 
with respect to a specific example. 

First, with respect to each image to be searched, 

25 after dividing the image into, for example, 4x4 = 16 

blocks, extract color constituent information composed 
of a representative color of an individual block which 



represents the block and its constituent rate to assume 
a set of the information to be a color distribution 
features of the original image. By this manner, a color 
distribution features of each image to be searched is 
5 extracted and stored in advance in the color 

distribution features storage unit 61. 

Also with respect to an inquiry image, after 
dividing the image into, for example, 8x8 = 64 blocks, 
calculate a mean value of each individual block to 

10 generate a reduced image having 8 pixels by 8 pixels and 

subject the reduced image to DCT to regard an obtained 
coefficient as a frequency distribution features of the 
inquiry image. 

Here, for the search for an image whose feature 

15 is similar to an inquiry image by using a frequency 

distribution features of the inquiry image, data of the 
color distribution features stored in the color 
distribution features storage unit 61 will be converted 
into frequency distribution information and both images 

20 will be compared using their frequency distribution 

features to calculate their similarities. 

For this purpose, the image search system first 
calculates a representative color of each block from a 
color constituent features of each block included in the 

25 color distribution features read from the to-be-searched 

image color distribution features storage unit 61. The 
representative color can be calculated by multiplying 
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each color by its rate and adding all the multiplication 
results . 

Then, generate an image having 4 pixels by 4 
pixels which is composed of the calculated 
5 representative colors of the respective blocks. 

Next, expand the image of 4 pixels by 4 pixels to 
an image of 8 pixels by 8 pixels whose image size is a 
target of frequency transformation at the time of 
generating a frequency distribution features of the 
10 inquiry image . 

Next, after dividing the image into, for example, 
8 x 8 = 64 blocks, calculate a mean value of an 
individual block to generate a reduced image of 8 pixels 
by 8 pixels, and subject the reduced image to DCT to 
15 extract the obtained coefficient as a frequency 

distribution features of the image to be searched. 

As a result, since an inquiry image and an image 
to be searched can be compared with respect to feature 
of the same dimension, search is realized by comparing 
20 the extracted frequency distribution features and the 

frequency distribution features of the inquiry image to 
calculate their similarities. 

As described in the foregoing, the image search 
system of the present embodiment enables calculation of 
25 a similarity between a frequency distribution features 

applied as an image features for use in inquiry and a 
frequency distribution features obtained by converting a 



color distribution features of an image which is an 
image features. This enables the same similarity 
calculation unit to handle both a color distribution 
features and a frequency distribution features to reduce 
5 the scale of the search system structure. 

Furthermore, an image search system can be 
provided which is capable of conducting search 
regardless whether a kind of image features is a 
frequency distribution features or a color distribution 

10 features. 

[FIFTH EMBODIMENT] 

Fig. 8 is a block diagram showing a structure of 
an image search system according to a fifth embodiment 
of the present invention. With reference to Fig. 8, the 

15 image search system of the present embodiment includes 

the to-be-searched image color distribution features 
storage unit 61, color distribution features conversion 
units 3 0 and 30a and the frequency distribution 
similarity calculation unit 42. The color distribution 

20 features conversion units 30 and 30a include 

representative color determination units 31 and 31a, 
image generation units 32 and 32a, image size change 
units 33 and 33a and frequency distribution features 
extraction units 34 and 34a. 

25 The to-be-searched image color distribution 

features storage unit 61 stores, as an image features of 
an image to be searched, a color distribution features 
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which is a set of color constituent feature of 
respective blocks obtained by dividing the image to be 
searched into a plurality of blocks. 

In the color distribution features conversion 
5 unit 30 on the side of the color distribution features 

storage unit 61, the representative color determination 
unit 31 calculates a representative color of each block 
from a color constituent features of each block included 
in the color distribution features read from the to-be- 

10 searched image color distribution features storage unit 

61. Here, the representative color can be set to be a 
mean value of a color contained in the color constituent 
features. Calculation can be made by multiplying each 
color by a rate of the same and adding all the 

15 multiplication results. At the time of calculation of a 

mean value, the mean value may be obtained by converting 
each color into a different color space. Possible in 
addition to those mentioned above is using a mode, a 
medium value etc. as a representative color. 

20 The image generation unit 32 generates an image 

composed of the representative colors of the respective 
blocks calculated by the representative color 
determination unit 31. The image size change unit 33 
changes the size of the image generated by the image 

25 generation unit 32. The image size can be made 

coincident with the size of an image as the original at 
the generation of the frequency distribution features. 
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The frequency distribution features extraction unit 34 
conducts frequency-transformation of the image obtained 
by the image size change unit 33 to generate a frequency 
distribution features. 

In the color distribution features conversion 
unit 30a on the side of receiving input of an inquiry 
image, the representative color determination unit 31a 
calculates a representative color of each block from a 
color constituent features of each block contained in 
the color distribution features of the inquiry image. 
Here, the representative color can be set to be a mean 
value of a color contained in a color histogram. 
Calculation can be made by multiplying each color by a 
rate of the same and adding all the multiplication 
results. At the time of calculation of a mean value, the 
mean value may be obtained by converting each color into 
a different color space. In addition to those mentioned 
above, it is possible to use a mode, a medium value etc. 
as a representative color. 

The image generation unit 32a generates an image 
composed of the representative colors of the respective 
blocks calculated by the representative color 
determination unit 31a. The image size change unit 33a 
changes the size of the image generated by the image 
generation unit 32a. The image sizes can be made 
coincident with each other. The frequency distribution 
features extraction unit 34a conducts frequency- 
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transformation of the image obtained by the image size 
change unit 33a to generate a frequency distribution 
features . 

The frequency distribution similarity calculation 
5 unit 42 calculates a similarity between the frequency 

distribution features generated by the color 
distribution features conversion unit 30 and the 
frequency distribution features generated by the color 
distribution features conversion unit 30a. 

10 By thus sequentially reading color distribution 

features of the plurality of images to be searched which 
are stored in the to-be-searched image color 
distribution features storage unit 61 to calculate a 
similarity between a frequency distribution features 

15 obtained by converting the read color distribution 

features and a frequency distribution features obtained 
by converting the color distribution features of the 
inquiry image, thereby selecting an image to be searched 
according to the degree of the calculated similarity, an 

20 image similar to the inquiry image can be specified from 

among the plurality of images to be searched whose color 
distribution features is stored in the to-be-searched 
image color distribution features storage unit 61. 

As described in the foregoing, the image search 

25 system of the present embodiment enables calculation of 

a similarity between a frequency distribution features 
converted from a color distribution features of an image 



- 55 - 



which is applied as an image features for use in inquiry 
and a frequency distribution features converted from a 
color distribution features of an image which is an 
image features. Therefore, even when no function of 
calculating a color distribution similarity is provided, 
this enables comparison of color distribution features 
by using the function of calculating a frequency 
distribution similarity. 

Furthermore, an image search system can be 
provided which is capable of conducting search 
regardless whether a kind of image features is a 
frequency distribution features or a color distribution 
features . 

[SIXTH EMBODIMENT] 

Fig. 9 is a block diagram showing a structure of 
an image search system according to a sixth embodiment 
of the present invention. With reference to Fig. 9, the 
image search system of the present embodiment includes 
the to-be-searched image frequency distribution features 
storage unit 62, the color distribution features 
conversion unit 30a and the frequency distribution 
similarity calculation unit 42. The color distribution 
features conversion unit 3 0a includes the representative 
color determination unit 31a, the image generation unit 
32a, the image size change unit 33a and the frequency 
distribution features extraction unit 34a. 

The to-be-searched image frequency distribution 
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features storage unit 62 stores a frequency distribution 
features of an image which is an image features of an 
image to be searched. 

In the color distribution features conversion 
5 unit 30a, the representative color determination unit 

31a calculates a representative color of each block from 
a color constituent features of each block contained in 
a color distribution features of an inquiry image. Here, 
the representative color can be set to be a mean value 
10 of a color contained in the color constituent features. 

Calculation can be made by multiplying each color by a 
rate of the same and adding all the multiplication 
results. At the time of calculation of a mean value, a 
mean value may be obtained by converting each color into 
15 a different color space. In addition to those mentioned 

above, it is possible to use a mode, a medium value etc. 
as a representative color. 

The image generation unit 32a generates an image 
composed of the representative colors of the respective 
20 blocks calculated by the representative color 

determination unit 31a. The image size change unit 33a 
changes the size of the image generated by the image 
generation unit 32a. The image size can be made 
coincident with each other. The frequency distribution 
25 features extraction unit 34a conducts frequency- 

transformation of the image obtained by the image size 
change unit 33a to generate a frequency distribution 
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features . 

The frequency distribution similarity calculation 
unit 42 calculates a similarity between the frequency 
distribution features generated by the color 
5 distribution features conversion unit 30a and a 

frequency distribution features read from the to-be- 
searched image frequency distribution features storage 
unit 62 . 

By thus sequentially reading frequency 
10 distribution features of the plurality of images to be 

searched which are stored in the to-be-searched image 
frequency distribution features storage unit 62 to 
calculate a similarity between the frequency 
distribution features and a frequency distribution 
15 features obtained by converting the color distribution 

features of the inquiry image, thereby selecting an 
image to be searched according to the degree of the 
calculated similarity, an image similar to the inquiry 
image can be specified from among the plurality of 
20 images to be searched whose frequency distribution 

features is stored in the to-be-searched image color 
distribution features storage unit 62. 

As described in the foregoing, the image search 
system of the present embodiment enables calculation of 
25 a similarity between a frequency distribution features 

obtained by converting a color distribution features of 
an image which is applied as an image features for use 



in inquiry and a frequency distribution features of an 
image which is an image features. This enables the same 
similarity calculation unit to handle both color 
distribution information and a frequency distribution 
5 features, which allows reduction in a scale of a 

structure of the search system. 

Furthermore, an image search system can be 
provided which is capable of conducting search 
regardless whether a kind of image features is a 
10 frequency distribution features or a color distribution 

features . 

Next, description will be made of an experiment 
on search precision conducted for the search system of 
the present invention. 

15 Experiment of the search processing by the search 

system of the present invention is here conducted using 
a data base which records data of 5466 sheets. 

On this occasion, for each of 50 sheets of 
inquiry images, a similar image is selected and set as a 

20 right answer in advance by subjective evaluation. The 

images are arranged in descending order of similarity in 
the image features between an inquiry image and images 
included in the data base and the order of right images 
of higher ranks down to the n-th place and the number of 

25 the same are evaluated as "normalized mean search order" 

to evaluate search performance. Normalized mean search 
order is assumed to be a value obtained by normalizing, 
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between 0 and 1, a value obtained by subtracting a mean 
value of an order in a case where all the right images 
are detected at the highest place from a mean value of 
an order of higher-ranking right images detected within 
5 the n-th place, and the smaller the value is, the higher 

the performance will be. "n" is assumed to be four times 
the number of right images selected in advance and right 
images not detected within the n-th place are all 
handled as being detected in the (n+l)th place. 
10 The following two kinds of image features are 

prepared. 

As a first image features, a frequency 
distribution features of an image is used. Generation 
method is as follows. Divide a still image into 8x8 

15 blocks and calculate a mean color of each block to 

generate a reduced image of a fixed size (8 pixels by 8 
pixels). Next, execute discrete cosine transform (DCT) 
with respect to the reduced image to extract a low- 
degree coefficient string of a luminance signal and a 

20 color difference signal. Furthermore, convert the 

obtained DCT coefficient using a conversion table to 
obtain a freguency distribution features A. 

Used as the other image features is a color 
distribution features . Divide a still image into 8x8 

25 blocks and extract color constituent information from 

each block and use the same. From each block, extract 
one or a plurality of representative colors and its or 
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their frequencies to obtain a color distribution 
features A. 

Next, convert the frequency distribution features 
A (DCT coefficient) to a color distribution features B 
5 by the following manner. 

First, inverse-orthogonal-convert the DCT 
coefficient of the frequency distribution features A to 
generate an image of 8 x 8 pixels. Then, with each pixel 
of the image having the 8x8 pixels regarded as a block, 

10 extract a color of a pixel and its rate (100%) as color 

constituent information from each block. A set of the 
extracted color constituent information is referred to 
as the color distribution features B. 

In addition, convert the color distribution 

15 features A to a frequency distribution features B (DCT 

coefficient) by the following manner. 

First, obtain a product of a rate of a color 
frequency and a pixel value of each block with respect 
to each representative color and assume a total sum to 

20 be mean color information. Next, generate an image of 8 

x 8 pixels composed of a mean color of each block. With 
respect to the generated image of 8 x 8 pixels, conduct 
DCT to extract a low-degree coefficient string of a 
luminance signal and a color difference signal out of 

25 the obtained coefficients. Furthermore, convert the 

obtained DCT coefficient using a conversion table to 
generate the frequency distribution features B. 



With the data size of the frequency distribution 
features A and the frequency distribution features B and 
the data size of color distribution features A and the 
color distribution features B set to be substantially 
5 the same, a normalized mean search order is calculated 

which is obtained when search is conducted using the 
respective image features. 

Table 1 shows a normalized mean search order 
obtained when using the frequency distribution features 
10 as an image features, while Table 2 shows a normalized 

mean search order obtained when using the color 
distribution features as an image features. 



[Table 1] 



COLOR DISTRIBUTION 
FEATURES 


NORMALIZED MEAN 
SEARCH ORDER 


A 


0.324813 


B 


0.414585 


[Table 2] 


FREQUENCY 
DISTRIBUTION FEATURES 


NORMALIZED MEAN 
SEARCH ORDER 


A 


0.209047 


B 


0.209288 
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It can be seen from Table 1 that as compared with 
a case where a color distribution features is generated 
from an original image, when using a color distribution 
features generated by the conversion from a frequency 
5 distribution features, the normalized mean search order 

obtains about three-fourths the performance. It can be 
seen from Table 2 that as compared with a case where a 
frequency distribution features is generated from an 
original image, a normalized mean search order when 

10 using a frequency distribution features generated as a 

result of conversion of a color distribution features is 
substantially coincident to barely have the danger of 
deteriorating search performance. 

It is therefore clear that both the color 

15 distribution features and the frequency distribution 

features can be handled by the same similarity 
calculation unit and that an image search system whose 
system structure is simplified without largely 
deteriorating search performance can be provided. 

20 Furthermore, an image search system can be 

provided which is capable of conducting search 
regardless whether a kind of image features is a 
frequency distribution features or a color distribution 
features . 

25 Fig. 10 is a block diagram showing a structure of 

other embodiment of the image search system of the 
present invention. 



In the example of the image search system shown 
in Fig. 10, a search processing unit lOOf receives input 
of an image features of an inquiry image from an inquiry 
image features input unit 5 0 and refers to an to-be- 
5 searched image features storage unit 60 for an image 

features of each image to be searched to determine and 
output a similarity of the image. 

The search processing unit lOOf includes a data 
processing unit 10 for converting a kind of image 

10 features and a similarity calculation unit 40 for 

calculating a similarity. 

Here, in the image search system as the example 
shown in Fig. 10, the data processing unit 10 has both 
the frequency distribution features conversion unit 2 0 

15 and the color distribution features conversion unit 3 0 

and also the similarity calculation unit 40 has both the 
color distribution similarity calculation unit 41 and 
the frequency distribution similarity calculation unit 
42 such that search can be properly conducted in either 

20 of a case where image features of an inquiry image and 

an image to be searched are frequency distribution 
features and a case where the same is a color 
distribution features. 

The data processing unit 10 then appropriately 

25 switches an image features applied from the inquiry 

image features input unit 50 and an image features 
obtained with reference to the amount of to-be-searched 
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image features storage unit 60 by means of a switching 
unit 11 to convert an image features when conversion is 
required and otherwise transfer the amount to the 
similarity calculation unit 40 without conversion. 
5 Then, also at the similarity calculation unit 40, 

the switching unit 43 appropriately switches the image 
features transferred from the data processing unit 10 
and transfers the amount to either one of the color 
distribution similarity calculation unit 41 and the 

10 frequency distribution similarity calculation unit 42 

that corresponds to the kind of image features to 
calculate a similarity and output the same. 

Therefore, the image search system of the present 
embodiment is allowed to freely handle both the color 

15 distribution features and the frequency distribution 

features, whereby an image search system can be provided 
which is capable of conducting search regardless whether 
a kind of image features is a color distribution 
features or a frequency distribution features . 

20 In addition, the above-described image search 

systems according to the respective embodiments have 
none of limitation of a kind of color in color 
constituent information to a specific color space and 
allows application to various color spaces such as RGB 

25 color space, YCbCr color space ad HSV color space. 

As a manner of determining a representative color, 
not all component colors are necessarily added and 
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evened and a method using a mean value of only a part of 
component colors is possible. 

Moreover, each unit such as the feature storage 
unit or the similarity calculation unit is not 
5 necessarily provided in the same device but may be 

disposed distributedly at a plurality of devices. Also 
possible is, for example, a mode in which data of an 
image to be searched is referred to through a 
communication network. 

10 In the above-described image search systems 

according to the respective embodiments, the functions 
of the data processing unit 10, the frequency 
distribution features conversion units 20 and 20a, the 
color distribution features conversion units 30 and 30a, 

15 the similarity calculation unit 40, the color 

distribution similarity calculation 41, the frequency 
distribution similarity calculation unit 42 and the like 
can be realized not only as hardware but also by loading 
a computer program having the respective functions into 

20 a memory of a computer processing device. The computer 

program is stored in recording media 90, 90a, 90b, 90c, 
90d, 90e and 90f such as a magnetic disc, a 
semiconductor memory or a CD-ROM. Then, loading the 
program from the recording medium into the computer 

25 processing device to control operation of the computer 

processing device realizes each of the above-described 
functions . 
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Although the present invention has been described 
and illustrated with respect to the preferred modes and 
embodiments in the foregoing, the present invention is 
not limited to the above-described modes and embodiments 
but can be implemented in variation within a scope of 
the technical ideas. 

As described in the foregoing, according to the 
image search system of the present invention and the 
image search method thereof, converting an image 
features to unify its kind enables an image similar to 
an inquiry image to be specified from among images to be 
searched regardless whether a kind of image features 
used is a frequency distribution features or a color 
distribution features, while realizing excellent search 
performance and allows the same similarity determination 
function to be applied to different kinds of image 
features, so that the structure of the image search 
system can be simplified. 

Although the invention has been illustrated and 
described with respect to exemplary embodiment thereof, 
it should be understood by those skilled in the art that 
the foregoing and various other changes, omissions and 
additions may be made therein and thereto, without 
departing from the spirit and scope of the present 
invention. Therefore, the present invention should not 
be understood as limited to the specific embodiment set 
out above but to include all possible embodiments which 



can be embodies within a scope encompassed and 
equivalents thereof with respect to the featureet out in 
the appended claims. 



